Novel Lossy Compression Algorithms with Stacked Autoencoders

Lossy compression is a strategy to reduce the size of data while maintaining the majority of its useful or meaningful information. Though some data is lost in the compression process, lossy compression algorithms try to remove data that is unlikely to be considered important by a human consumer. Given this goal, today’s lossy compression techniques are carefully designed to model human perception as closely as possible in order to maximize the amount of valuable retained information for a given data compression level. The popular JPEG image compression algorithm, for example, is reliant upon the observation that human visual perception is far more sensitive to low-frequency spatial variation than high-frequency spatial variation. Similarly, the MP3 audio compression algorithm uses carefully-tuned psychoacoustic models to make inferences about which components of a given audio stream are most perceived by a human listener. These perceptual coding methods exhibit quite impressive compression results, but require a great deal of work among psychologists, computer scientists, and engineers, and are ineffective or even detrimental when applied in situations for which they were not designed. Using JPEG to compress line art, for example, often yields undesirable results. It is thus valuable to be able to easily develop a compression method that is naturally tuned to a particular type of data. This can be accomplished by extracting statistical regularities in the data and then using this information to compress the data. We demonstrate that one can efficiently compress data through the careful application of deep-belief networks while maintaining the data’s most meaningful characteristics.